Optimal control models of the goal-oriented human locomotion 



Yacine Chitour* Frederic Jean| Paolo Mason* 



June 22, 2010 



Abstract 



In recent papers it has been suggested that human locomotion may be modeled as an inverse optimal 
control problem. In this paradigm, the trajectories are assumed to be solutions of an optimal control 
problem that has to be determined. We discuss the modeling of both the dynamical system and the cost 
to be minimized, and we analyze the corresponding optimal synthesis. The main results describe the 
asymptotic behavior of the optimal trajectories as the target point goes to infinity. 

Keywords: human locomotion, inverse optimal control, Pontryagin maximum principle, stable manifold 
theorem. 

1 Introduction 

Ask a person walking in a empty room to leave this room through a given door. Which path docs that person 
choose? The purpose of this paper is to deal with this issue, that is to understand the goal-oriented human 
locomotion. 

We follow the approach initiated in [TJ[2l|3]. In this framework, the locomotor trajectories lie in the simple 
3-D space of both the position (x, y) and the orientation 9 of the body. The problem we want to address is 
then the following one. Given an initial point (xo,yo;^o) and a final point (xi,?/i,#i) (see Figure [1]), which 
trajectory is experimentally the most likely? 

A nowadays widely accepted paradigm in neurophysiology is that, among all possible movements, the 
accomplished ones satisfy suitable optimality criteria (see [15] for a review). One is then led to make the 
assumption that the chosen locomotor trajectory is solution of some optimal problem, namely: minimize some 
integral cost 



among all "admissible" trajectories joining the initial point to the final one. Two questions are in order. First, 
what are the "admissible" trajectories? It is in particular necessary to precise both the dynamical constraints 
applied to the locomotion, and the regularity of the trajectories (and so the order of derivation entering in the 
cost function L). Secondly, how to choose the cost function? Once the set of admissible trajectories is defined, 
the latter question takes the form of an inverse optimal control problem: given recorded experimental data, 
infer a cost function L such that the recorded trajectories are optimal solutions of the associated minimization 
problem. 

In the theory of linear-quadratic control, the question of which quadratic cost is minimized in order to 
control a linear system along certain trajectories was already raised by R. Kalman |10j . Some methods allowed 
deducing cost functions from optimal behaviour in system and control theory (linear matrix inequalities, [7]) 
and in Markov decision processes (inverse reinforcement learning, [13]). However all these methods have 
been conceived for very specific systems and they are not suitable in the general case. A new and promising 
approach of the inverse optimal control problem has been dcvelopped in [5] [H] for the pointing movements of 
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Figure 1: Goal-oricntcd human locomotion. 



the arm. In that approach, based on Thorn transversality theory, the cost structure is deduced from qualitative 
properties highlighted by the experimental data. Thus, starting from the observation of inactivity intervals 
of the muscles during pointing movements, it has been proven that their presence is a sort of necessary and 
sufficient condition for the cost to be of a certain type (namely of "absolute work" type). 

The method chosen in the present paper bears some resemblance with JSJ[S]. However no strong qualitative 
properties such as the inactivations come out in the registered data, which prevent the use of transversality 
theory We then choose a more direct approach, consisting of three steps: 

1. Modeling step: use experimental observations to define the set of admissible trajectories and to reduce 
the class of acceptable cost functions L; as a result, we obtain a class of optimal control problems indexed 
by the cost function L. 

2. Analysis step: make a qualitative analysis of the optimal synthesis of the above-defined problems; the 
aim is mainly to exhibit properties characterizing the dependence on L of the synthesis. 

3. Comparison step: through a numerical study based on the characteristic properties exhibited above and 
a comparison with the trajectories experimentally recorded, determine which is the cost function L which 
best fits the experimental data. 

The last step is of different nature than the first two ones. It requires the processing of a large number 
of registered data together with a numerical study. In this paper we will then be concerned only with the 
first two steps, the last one being the object of a forthcoming study. Note that in the paper [5] we already 
performed a similar analysis for a simplified model. 

The modeling step will be addressed in Section [5] and comes up with an optimal control problem (OCP). 
It is based on experimental observations together with two modeling assumptions, (Al) and (A2), and also 
on technical hypothesis, (H1)-(H4). 

Section [3] is dedicated to the analysis step, all the technical proofs being postponed to Section @] We first 
show the existence of optimal solutions and apply the Pontryagin maximum principle to Problem (OCP). 
After a detailed analysis of the extremal flow, we show that the angle 9 along an optimal trajectory is solution 
of a fourth-order differential equation (OPT) p with parameter p. In case the initial point and the target are 
far enough one from each other, we are able to compute the asymptotic value p* of the parameter and to 
prove that the orbit corresponding to an optimal trajectory lies in the stable manifold of the (unique) unstable 
equilibrium of (OPT) p * . This asymptotic behavior exactly provides the type of characteristic properties of the 
trajectories of human locomotion we are looking for in order to test the adequacy of the model to registered 
data. The latter task will be the subject of a future work. 

Moreover, based on the above asymptotic property, one can device a numerical method for determining 
the initial value of the adjoint vector and thus completely integrating the extremal flow. From the point of 
view of optimal control, this geometric method is the main technical novelty of our paper. It belongs to the 
realm of indirect methods, but is clearly of different nature than the shooting methods. 
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Figure 2: The nonholonomic constraint. 

2 Optimal control based modeling 

The first task is to define what is the set of admissible trajectories. It is important here to precise the typical 
situation we want to model, namely a person entering a room through one door and leaving by another one. 
Hence, besides the fact that initial and final positions and orientations are fixed, the velocity is positive at the 
extremities (the person walks in and out the room). Moreover the two doors, and so initial and final positions, 
are supposed not too close one to the other. 

Our modeling is based on three important experimental observations made in [51 [3] . 

(i) The velocity is perpendicular to the body (see Figure^. Namely, denoting by 9 the orientation of the 
torso and by (x, y) the horizontal projection of the center of mass, the locomotion is submitted to the 
classical nonholonomic constraint: 



x sin 9 — y cos 9 = 0, i.e. 



x = v cos 9 
y = v sin 9 



where v = \J x 2 + y 2 is the tangential velocity. 



(ii) The tangential velocity has a positive lower bound: v > a > 0. As a consequence, all locomotion 
trajectories can be parametrized by the arc-length. We are interested here in the geometric curves of 
the human locomotion, not by their time-parametrized trajectories. We will then use the arc-length 
parametrization, which amounts to set v = 1. 

It is worth to notice that, on recorded data, the tangential velocity is generally almost constant along 
the trajectory. Thus, up to rescaling, arc-length and time parametrization approximately coincide. 

(Hi) The curvature varies continuously. That is, n(t) = 9(t) is a continuous function. 

It results from these observations that every locomotion trajectory satisfies 

x = cos 9, y = sin#, 9 = n, 

where k is a continuous function. Thus x, y and all their derivatives only depends on 9 and its derivatives. To 
complete the description of the set of admissible trajectories, it only remains to precise in which functional 
space is chosen the function 9. Remind that our main modeling assumption is that human locomotion is 
governed by optimality criteria. It is then mandatory to choose a functional space for the trajectories together 
with a class of cost functions so that one has general results ensuring the existence of optimal solution under 
reasonable hypothesis. As for the functional space, it must be complete and the natural choice is that of the 
Sobolev spaces W k ' p . In other words, we assume that the locomotion trajectories are solutions of the control 
system 

x = cost 
y = sin ( 
0(k) 

where u is a measurable function of finite LP norm, for a certain integer p > 0, and ft > is an integer. As 
for the cost functions L, the highest order of derivatives of 9 actually appearing in L must be equal to k. The 
observation (Hi) above suggests that the order k is at least equal to two. We will make here the assumption 
that it is exactly equal to two. 
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(Al) The cost function L = L(x, y, 8, 8, 8) explicitly depends on 9, and the latter has a finite L p norm. 



To sum up, the control system modeling the dynamics of the locomotion writes as: 




sin 



cos 
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(2.1) 



where X = (x, y, 9, k) belongs to M 2 x S 1 x R and the control u is a measurable function defined on an interval 
[0, T], where T > depends on u, and taking values in R. 

As for the initial and final points Xq,Xi, the spatial and angular components (x,y,9) are given by the 
data of the goal-oriented locomotion problem we consider. Two reasonable conditions could be assumed on 
the curvature: 

• cither k is let free, the cost C(u(-)) being minimized among all the points Xq,Xi with fixed spatial and 
angular components; 

• or it satisfies k = at Xq,X\ (the trajectory starts and ends in straight line). 

This second hypothesis corresponds to the particular experimental setting considered in [2]. In this paper 
we will follow this assumption, however the other one essentially leads to the same results and we will mention 
in Section [3751 how they differ. 

The locomotion trajectories between two arbitrary configurations (2o,Z/Oj#o) and (xi, yi, 0i) then appear 
as the solutions of an optimal control problem of the following form: minimize the cost 



among all admissible controls u(-) steering System |2~T1 from X(0) = (xo,yo,9o,0) to X(T) = (x\, yi, 9i, 0). 
Notice that the time is free in this problem, T depends on u(-). 

The inverse optimal control problem consists in finding the proper cost function L. Some elementary 
remarks allows us to reduce the class of the candidates. 

(i) The cost function is defined in an egocentric frame; the whole problem is then invariant by rototransla- 
tions, and L is independent of (x,y,8), that is, 



(ii) Turning right or left are equivalent, hence the symmetry of L : L(—k, —u) = L(k,u). 

(iii) The more one turns, the more it costs. In other words, the partial functions K i— > L(k, u) and u i— > L(k, u) 
are non-decreasing function of |k| and |u| respectively. 

(iv) Going in straight line has a positive cost, L(0,0) > 0, which is the unique minimum of L. Since L is 
defined up to a multiplicative constant, we can impose the normalization: L(0,0) = 1 and L(n,u) > 1 
for (k, u) ^ 0. As a consequence, the cost of a straight line is its Euclidean length. 

(y) As already mentioned, the optimal control problem must have a solution, which requires some convexity 
properties on L. 

In this paper, we will restrict the class of cost functions satisfying the following assumption: 
(A2) The cost function L depends separately on k and u. 




L = L(k, u). 
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We will then consider a cost 

C(u(-))= / [1 + ?(«(*)) + V(u(i))]dt. (2.2) 
./o 

This cost appears as a compromise between the total time T (equivalently, the length to be covered) and an 
"energy term" depending separately on k and u. 

To satisfy the properties listed above we assume that the functions ip and ip verify the following technical 
hypotheses. 

(HI) <p and ip are non negative, C 2 and even functions defined on R, and are non decreasing on R + . Moreover, 
tp{0) = V(0) = 0; 

(H2) ip is strictly convex and ip"(0) > 0; 

(H3) there exist p > 1 and two positive constants C, i? such that 

V>(r) > C\r\ p , for every rel such that |r| > iJ. (2.3) 

We next provide some explanations about these hypotheses. 

Hypothesis (HI) directly results from properties (ii)-(iv) and is structural to the model, except for the 
regularity assumption. However, if the cost was not differentiable, the phenomenon of "inactivation" would 
appear in the optimal trajectories (see [3]). Since inactivation is not observed in the registered data, one must 
assume some regularity property for the cost. Note that the C 2 regularity of ip and ip required in (HI) is 
actually needed just for the development of the numerical method of Section [3.41 while the qualitative results 
of Sections 13.21 and 13.31 arc valid even if we assume that <p and ip are just C 1 . 

The convexity and the growth condition provided by Hypotheses (H2)-(H3) arc classical assumptions 
which ensure Property (i>) (sec Proposition 13.11 and Remark 13.21 below). The technical assumption ip"(0) > 
ensures that, firstly the minimizers are locally unique; secondly they are solutions of an ODE (of pendulum- 
type); and thirdly one can recover at least numerically the initial value of the adjoint vector (in an approach 
based on the Pontryagin maximum principle). It is worth to notice that (H2)-(H3) define open conditions 
on the set of functions tp and ip satisfying (HI). This stability property is necessary due to the physiological 
nature of the cost. 

The purpose of Assumption (A2) is to ensure that Property (v) is satisfied with reasonable hypotheses like 
(H2)-(H3) (see, for instance, j!2j for a discussion about existence of minimizers under similar assumptions). 

To summarize, the optimal control problem modeling the goal-oriented human locomotion is the following 
one. 

(OCP) Fix an initial point Xo = (0, 0, tt/2, 0). For every final point of the form X\ = (x\,yi, #i, 0), 
for some (xi,yi,8i) 6 I 2 x S 1 , find the trajectories of (|2.ip steering the system from X to X\ 
and minimizing the cost (|2.2[) . where ip and ip verify hypothesis (H1)-(H4). 

3 Qualitative analysis of the optimal trajectories 
3.1 Notations 

In the following we will make use of the following notations. 

We will always assume without loss of generality that the difference ot\ — a-i between two angles a±,a2 € [0, 2ir] 
takes values in the interval [— it, tt]. In particular with this notation the modulus \a± — a.2\ is a continuous 
function of «i, ot2 taking values in [0, n]. 

Given a subset S of [0,T] we will denote its complement in [0,T] by S c . 

The Lebesgue measure of a set S will be denoted by fi(S). 

The scalar product in M 2 is denoted by (•,•). 

The symbol B(x,r) indicates the ball of radius r centered at x. 

As usual, given two subsets A, B of a certain vector space the set A + B is defined as 

A + B = {x + y : x € A, y G B}. 



5 



3.2 Existence of optimal trajectories 

The existence of solutions to problem (OCP) is guaranteed by the following result. 

Proposition 3.1. For every choice of Xq and X\ in R 2 x S 1 x R there exists a trajectory X(-) of (|2.ip . 

defined on [0,T], associated to some control u(-) and minimizing C(u(-)) among all the trajectories starting 
from Xq and reaching X± . 

Proof. Consider a minimizing sequence «„(■) : [0,T„] — > R, that is 

lim C(u n (-)) = inf C(u(-)) 

ra->oo u(-) 

where u n (-),u(-) steer the system from Xq to X±. Since C(u n (-)) is uniformly bounded, one easily deduces 
from Hypothesis (HI) and (H3) that T n and ||w n ||iPQo,T„]) must be uniformly bounded, and therefore, up 
to a subsequence, we can assume that T n converges to T. Let us extend u n (-) on [0, oo) by taking u n (t) = 
if t > T n . Then, still up to subsequences, u n {-) converges to some u(-) G L p ([0,T]) in the weak topology 
of L p ([0,T}). Moreover, the functions K n (t) = f*u n (s)ds are uniformly continuous by Holder's inequality 

and they converge pointwise to R(t) = f u(s)ds since u n (-) — v u(-) in LP. By Ascoli-Arzela theorem, K n 
converge to R uniformly on [0, TJ. Therefore, from the equation (|2.ip . we immediately get that the trajectories 
X n (-) corresponding to u n (-) converge uniformly to the trajectory X(-) corresponding to u(-) and, since these 
trajectories are uniformly equi-continuous, X(T) = lim n _>oo X n (T n ) = X±. Moreover, we deduce that the 
sequence ^ J T " <p(n n (t))dtj converges to ip(ji(t))dt. 

Let us denote Lq(u(-),T) = \jj(u(t))dt. It remains to prove that Lq(u(-),T) = ]im n ^ foo Lo(u n (-), T n ). 
For this purpose, we first observe that the convexity of ifj(-) immediately implies the convexity in L p ([0,T]) 
of the functional Lo(-,T). It is well-known that the weak lower semi-continuity of a convex functional is 
equivalent to its strong lower semi-continuity (see for instance [5J Corollaire III.8] ). From this fact, and since 
Lo(u n (-),T) < Lo(u n (-),T n ), to conclude the proof of the proposition it is enough to show that Lq(-,T) is 
strongly lower semi-continuous in L p ([0, T]). Let us consider a sequence (vk)k>i in £ p ([0, Tj) converging in the 
LP norm to a function v. In particular we can extract a subsequence (i>&-)j>i converging almost everywhere 
to v and such that lim^oo L {vkj {■), T) = liminffc_ ) . 00 L (vk(-), T). According to Egoroff theorem there exists 
a sequence (Ah)h>i of open nested subsets of [0, T] such that lim^oo n(Ah) = and the convergence Vk — > v 
is uniform in [0, T] \ Ah- We get 



L (v(-),T) = lim (L (v(-),T)- / ^(v(t))cIt 



= lim lim [ L (v k A-),T) - / ip(v k jT))dr 



< 



lim L (v kj (-),T) 



which concludes the proof of the proposition. □ 

Remark 3.2. The argument we provided is inspired from the proof of Theorem 8, p. 209 of [12] . Let us 
stress that it is not difficult to show through appropriate examples that the convexity of ip required by 
Hypothesis (H2) is crucial for the existence of a minimizer. 



3.3 Application of the Pontryagin maximum principle 

In order to apply the classical Pontryagin maximum principle (PMP) [14] . one needs to know that the optimal 
control u(-) is bounded in the L°° topology. At the present stage of the analysis, we do not possess that 
information and we therefore must rely on more sophisticated versions of the PMP. For instance, one readily 
checks that (OCP) meets all the hypotheses required in Theorem 2.3 of [4] and we get the following. 

Proposition 3.3. Let X(-) be an optimal trajectory for (OCP), defined on [0,T] and associated to the control 
u(-). Then this trajectory satisfies the PMP. 
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Let us associate to System (|2.1[) the Hamiltonian function 



H = H{X,p, u, v) = pi cos 9 + P2 sin 9 + p^n + p±u — v{Y + ip(K) + tp(u)), (3-1) 

where p = {pi,P2,P3,P4) G R 4 is the adjoint vector and vel. 

The PMP writes as follows. Let u(-) be an optimal control defined on the interval [0,T] and X(-) the 
corresponding optimal trajectory, whose existence is guaranteed by Proposition 13. f I Then X(-) is an extremal 
trajectory, i.e. it satisfies the following conditions. There exists an absolutely continuous function p : [0, T] —> 
R 4 and v > such that the pair (p(-), v) is non-trivial, and such that we have: 



x(t) = f§(x(t), P (t),v, u m ,„ 9 , 

dX 



\p(t) = -^(X(t),p(t),u,u(t)) 



The maximization condition writes: 

H(X(t),p(t),u(t),v) = max H(X(t),p(t),v,u) for a.e. t e [0,T]. (3.3) 

As the final time is free, the Hamiltonian is zero (see [15]): 

H(X(t),p(t),u(t),u) = 0, Vte[0,T]. (3.4) 
The equation on the covector p, also called adjoint equation, becomes: 





= 0, 


P2 






= pi sin 


.P4 


= ~P3 4 



(3.5) 



If ^ 7^ we can always suppose, by linearity of the adjoint equation, that v = 1. In this case (resp., if 
v = 0) a solution of the PMP is called a normal extremal (resp., an abnormal extremal). It is easy to see that 
all optimal trajectories are normal extremals. Indeed, if v — 0, then p4 = by the maximization condition 
(|3.3p . From ^4 = 0, we immediately deduce that P3 = and, from p^ = 0, it remains p\ sin 9 — pi cos 9 = 0. 
From H = 0, one also has p\ cos 9 + P2 sin 6* = and thus p\ = P2 = 0. That contradicts the non-triviality of 
ip,v). 

Consequently, Equation (|3.4p becomes 

Pi cos 6* + pi s'm.9 + p 3 K + p 4 u — (1 + <^(k) + ?/>(u)) = 0. (3.6) 

As regards the maximization condition (|3.3p . the optimal control is given by 

u opt {t) = (V') _1 (P4(t)) forte [0,T]. (3.7) 

Note that the strict convexity and the growth condition on ijj imply that ijj' realizes a bijection from R to R 
and thus its inverse is a continuous and strictly increasing function from R to R. 

From (|3.5p . we get that pi and pi are constant. Let us write {p\,pi) = p(coscj), sin</>) for some (j> & [0, 2tt) 
and p > 0. Therefore from the Hamiltonian system (|3.2p we get that, along an optimal trajectory, the following 
equation, independent of u, is satisfied for a.e. t <G [0,T] and for a suitable choice of (p, </>) G R x [0,27r), 

= 

p 3 = psm{9 - 0), 

Note that this equation implies that the function 9 is solution of a fourth order differential equation of pendulum 
type depending on the parameters (p,4>). 

Setting (xi,yi) = \(xi, j/i)|(cos a, sina) for some a £ [0,2ir), a careful analysis of this equation yields the 
following results. 
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Proposition 3.4. There exists R > and C > such that, for every optimal trajectory with \(x\,yi)\ > R 
one has 

||(«,P3,P4)|U~ <C and \\u\\ L °°<C. 
Remark 3.5. The boundedness on ||/c||l°° is valid without any hypothesis on |(xi,yi)|. 

Proposition 3.6. For every r\ > there exists R^ > such that for every optimal trajectory with \(xi,yi)\ > 

\<p — a\ < r/ and |/? — 1 j < 77 



Rrf one has 



Theorem 3.7. Let us associate to any extremal trajectory X(-) o/(OCP) the function Z (t) = (6(t), K,(t),p3(t),p4,(t)). 
Given v > there exist t v > and er„ > 2r„ suc/i that, for every optimal trajectory with final time T > o~ v , 
one has 

\Z(t) - (a, 0,0, 0)| < v forte [t v ,T-t v ]. 



The proof of these results is rather long and technical, it is postponed to Section POl for Propositions 
and 13.61 and Section I4T41 for Theorem 13.71 

3.4 Numerical study of the asymptotic behavior 

The main information provided in the previous section concerning optimal trajectories, solutions of (OCP), 
can be summarized as follows: 

(1) if the norm of the spatial components {x\,y\) of X% is large enough then optimal trajectories can be 
decomposed in three pieces corresponding to time intervals [0,t], [i, T — t\, [T — i,T], where t can be 
thought independent of X\ and the arc of the trajectory on [t, T — t] is approximately a segment (the 
accuracy of the approximation depends on the size of t); 

(2) for any optimal trajectory there exist two scalars (p,<j>) such that Z(-) = (#(•), , P3(-) , P4(-)) satisfies 
Equation (|3.8[) . Also, the relation 



H{e,K,pz, PA ) := pcos{6 -cp)+ P3 K+p^')- 1 (Pi) - 1 - ¥>(«) - ^T l {Pi)) = (3.9) 

holds along the trajectory. Moreover, if (xi, yi) is large enough, p is close to 1 and (j> is close to the angle 
a such that (xi,yi) = \(xi, j/i)|(cosq!, sina). 

The qualitative properties stressed above do not allow neither to understand the local behavior of optimal 
trajectories, in particular on the intervals [0,i], [T — i,T] defined by the above Condition (1), nor to find them 
numerically. However they detect some non-trivial asymptotic behavior of the pair (p, </>) and of the initial 
data of (|3.8[) . for large values of (xx,yi). The analysis carried out in this section arises from the observation 
that, in order to understand the asymptotic shape of the optimal trajectories on [0,t\, [T — i,T], it would 
be enough to complete the information about the initial data of (|3.8[) . Indeed, as far as the initial datum of 
the equation is close to its asymptotic value (if it exists) and (p,4>) is close to (l,a), we know, from classical 
continuous dependence results for the solutions of differential equations, that the solution of (|3.8D . will in turn 
be close (on compact time intervals) to the solution of the asymptotic equation 

e = K, 

« = W)-*M ( 3.io) 

pz = sm{8 - a), 
Pa = -Pa + <^'( K )- 

where we take as initial value the asymptotic value of the initial data for (|3.8[) . In other words, a precise 
knowledge of the asymptotic behavior of such initial data, for large (xi,yx), would provide a tool to study 
numerically, through (|3 . 10[) . the asymptotic shape of optimal trajectories on [0,t\ (and, by symmetry, on 
[T — t,T]). 
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Figure 3: Behavior around the stable and unstable manifolds. 



Let us first notice that an asymptotic value for P4(0) is simply provided by evaluating (|3.9[) at time 0, with 
the approximation (p, (f>) = (1, a). More precisely pi(0) coincides with a solution z of the equation 



cos(tt/2 - a) + z (VO^O) - 1 - i){{^')~ l {z)) = . 



(3.11) 



Since the map -q i-> ip'iv) 7 ! ~ ^(v) = foi^'iv) ~ ^'(aO) dp is strictly increasing for rj > 0, strictly decreasing 
for negative r\ and goes to infinity for \rj\ going to infinity, because of the strict convexity of tp, and since 
V> € C 1 , we know that the previous equation has exactly one positive solution and one negative solution. Since 
u(0) = (tp')' 1 (p4(0)), this suggests the existence of two asymptotic behaviors for the trajectories of (|3.8[) . each 
one corresponding to a candidate solution for (OCP). These two trajectories start from Xq by turning on 
opposite directions. 

To complete the information about the asymptotic value of the initial data for (|3.8p we need to investigate 
the possible values of j?3(0). For this purpose we will develop below a numerical method based on the existence 
of a stable manifold for (|3.10p . 



An equilibrium for (|3.10[) is given by (9, K,p3,p4) = (a, 0,0,0) and we know from Theorem 13.71 that, for 
solutions of (OCP) with (x±, yi) far enough from the origin, the corresponding values of (0(-), k(-),P3(-),P4(-)) 
are close to this equilibrium on some interval [i,T — t] for large i and T, which suggests some stability property 
of the equilibrium. It is actually easy to see that Z eq = (a, 0, 0, 0) is not a stable equilibrium of the system. 
Indeed the linearized system around Z eq is 



Z = J(Z - Z eq ) , J = 



/ 



1 

V 



1 




<p"(0) 







-1 





W(o) 
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ze 



(3.12) 



where the matrix J has exactly two eigenvalues Ai, A2 with negative real part, corresponding to some eigen- 
vectors Vi,V2, while the other two eigenvalues p\,^2, with corresponding eigenvectors w\,W2, have positive 
real part. Therefore Z eq is a stable equilibrium for the linearized dynamics restricted to Z eq + V , where V is 
the two dimensional real subspace of K 4 spanned by V\ , V2 (notice that V\ , V2 can be assumed either real or 
complex conjugate). 

The classical stable manifold theorem (see for instance [TT]) ensures the existence of a manifold W s of 
dimension 2, called stable manifold, which is tangent to V and which contains all the trajectories converging 
to the equilibrium (exponentially fast). Note that, since the continuous function H, with (p,<p) = (l,a), is a 
first integral of the dynamics ([37X01) and H{Z eq ) = 0wc have W s C H^ 1 ^)- 

On a small neighborhood of the equilibrium all the trajectories that are not contained in W s diverge from 
it exponentially fast (see Figure [3]). Let us fix such a neighborhood U. From Theorem 13.71 we know that 
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Figure 4: Asymptotic behavior of optimal trajectories with final point far from the origin. 



there exists t such that, if Z(-) is a trajectory of p.8[) associated to a solution of (OCP), then Z(t) E U 
for every t G [i, T — t\, provided that (x±,yi) is far enough from the origin. In particular if we consider a 
sequence of final points x[ n ^ for (OCP) with spatial components (x^, y^) = n(cos a, sin a) we deduce that, 
for the corresponding sequence of trajectories Z^ n '(-), the limit Z of Z^(t) exists (up to a subsequence) and 
is contained in W s . Continuous dependence results for the solutions of differential equations guarantee that 
the limit of Z^ n \0) coincides with Z(0), where Z(-) is the solution of (|3.10|) such that Z(t) = Z. In particular 
it must be Z(0) = (tt/2, 0,p 3 ,p4) where p4 satisfies (|3.11l) . 

The previous reasoning suggests a method to study numerically the possible values of p 3 at time 0. Indeed if 
U is small enough then W s is well approximated by the affine space Z eq + V. Consequently one can numerically 
look for solutions of the asymptotic equation (|3.10j) with 

z(t) e (z eq + v)nu 

and such that 8(0) = tt/2, k(0) = 0. More precisely a simple numerical method can be specified as follows. Let 
us fix a closed curve 7(5) = s (cos(s)wi + sin(s)v2), where v\,V2 are real vectors spanning V and e is a small 
constant (the precision of the method increases as e goes to zero) . Since all the trajectories converging to the 
equilibrium must cross this curve (in the approximation W s ~ Z eq + V) we can recover them by following 
backwards in time the solutions of (|3.10j) starting at Z(0) = Z eq + j(s) up to a time t < such that = 0. 
The candidate approximate asymptotic trajectories we are looking for are then determined by the values of 
s for which, for a reasonably not too large \t\ such that n(t) — 0, we also have 0(i) = tt/2. The value Z(0) 
is then a candidate value for the initial datum of a trajectory of (|3.8p associated to a solution of (OCP), for 
large values of (xi, y\). Moreover this simple method allows to approximate numerically the initial arc of such 
optimal trajectories (see Figure S] which considers the case tp = 0, ip(z) = z 2 ). 

An effective method to globally construct solutions of (OCP) for large values of (x\,y{) is the following. 
Define a further closed curve 7(5) = e (cos(s)wi + sin(s)w 2 ), where W\, W2 are real vectors generating the 
unstable subspace W (defined similarly to V). Assume that e <C £ <C 1 and consider the solutions of (|3.8[) 
with <j) = and starting from Z(0) = j(s) + j(s), for suitable choices of p,£,i, s, s such that H(Z(0)) = 0. 
For a fixed small enough e > and fixed s 6 [0, 2tt] it turns out that the trajectory on intervals [fi, 0], with 
t\ < not too large, is subjected to small variations with respect to the choice of e -C s,s £ [0,2ir],p such 
that H(Z(0)) = 0. In other words the trajectory approximately only depends on s,s on the interval [ii,0]. 
Similarly as before, the value s and the time t\ can be chosen in such a way that tt(ti) = and, at the same 
time, 0(ti) = tt/2 — <fi, for a prescribed value <j>. 

On the other hand for positive time the components along the stable subspace V decrease exponentially 
as far as the components along W are small so that, after a certain time, the trajectory evolves close to the 
unstable manifold W u (see Figure [3]) . The dynamics at this stage essentially depends on the initial choice of 
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Figure 5: Candidate optimal trajectories for large values of (x\,y{). 

e and s, where the first parameter determines the range of time such that the trajectory is confined inside U, 
while the second one essentially determines the final angle. 

This method gives rise, up to a rotation of an angle <j) and appropriate translations, to solutions of (OCP). 
Figure [S] depicts a set of candidate solutions of (OCP) constructed by using the previous method, for a 
particular choice of the angle (f> and in the case <p = 0, 4>(z) = z 2 . 



3.5 The case with free curvature at the extremities 

In the definition of the optimal control problem (OCP) given in Section [2j we have chosen to take the initial 
and final values of the curvature equal to 0. Another reasonable condition would be to let these values free. 
In this case, the optimal control problem writes as follows. 

(OCP) Fix an initial point Qo = (0, 0,7r/2) in coordinates (x,y,9). For every final point Q\ 



'■xS 1 , find the trajectories X(-) = (x(-), y(-), #(•), «Q) of (EH sudl th &t (x(0),y(0), 6(0)) 



Qo and (x(T),y(T),6(T)) = Q x , and minimizing the cost ((2~2l) under hypotheses (H1)-(H4). 

The analysis of (OCP) leads essentially to the same results than the one of (OCP). The only noticeable 
differences are the following ones. 

• When applying the PMP, in addition to the Hamiltonian equations p. 21) , one obtain also a transversality 
condition on (p(0),p(T)), namely 

p 4 (0)=p 4 (T) = 0. 

These conditions play the same role than the condition k(Q) = k(T) = in (OCP). 

• In Section 13^41 the relation p.9[) on the Hamiltonian allows to characterize the asymptotic value k(0) 
(and not the one of P4(0) through Equation (|3.11l) as in (OCP)), in function of ^3(0). Indeed, since 
P4(0) = 0, one has, for the asymptotic values: 

cos(tt/2 - a) + P3 (0)k(0) - 1 - p(k(0)) = 0. 



The numerical methods presented in Section 13.41 have to be slightly modified in accordance with the 
changes above. 
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Figure 6: The function k(-) associated to the control function of Proposition |4~T1 
Note also that the proof of Lemma 14.81 has to be modified (see Remark 14.91) . 

4 Proofs of the main results 

4.1 Comparison with reference trajectories 

In order to obtain a first rough estimate of the optimal cost, we exhibit particular trajectories of (|2.1[) steering 
the system from X$ to X\ . 

Proposition 4.1. Given a > 0, a pair (Ao, Ai) S M3_ and T > 2Aq + 2Ai we define the control function 



—a 


t e [o, A ] 


+o- 


te (A ,2A ] 





te (2A ,T-2Ai] 


+cr 


te (T-2A X ,T-Ai 


— a 


<e (T-Ai,T] 



Then, for every choice of X\ = (x\,yx, 0\, 0) with \{x\,yi)\ > there exists a pair (Ao, Ai) <G [0, y^Sn/a] x 

[0, y/5Tv/cr] and T > 2Xq + 2Ai suc/i i/iai i/ie trajectory of (j2.1j) im£/i «(•) given by (|4.ip starting at Xq reaches 
X\ at time T . 

Proof. First, let us observe that k = along the arc corresponding to u = 0, which is therefore a segment, 
and k(0) = k(T) = (see Figure^. Also, since 9 = u, one can easily check that 9(t) — 9q — trAg between 2Ao 
and T — 2Ai and that 9(T) = 0q — crAg + a\\. Therefore, since it must be 9 = 9\ at the final time, and up to 
assuming without loss of generality that 9\ — 9q € [0,27r), we can assume that Ai is the following continuous 
function of Ao € [0, ^/3ir/a]: 

Al = A 1 (Ao) = y^^ + Ag. 

Let us denote by 7o(-) the trajectory of (|2.1[) corresponding to u(-) starting at Xo at time and by 7i(-) the 
trajectory of (|2.1j) corresponding to u(-) based at X\ at time T. Our aim is to prove that, for an appropriate 
choice of (Ao,? 1 ), these trajectories coincide. For this purpose it is enough to prove that their projections on 
the plane coincide. 

Let us consider the points Po and Pi which are the projections on the plane of 7o(2Ao) and 71 (T — 2Ai) 
(see Figure [7]). Since for both 7o(-) and 7i(-) the angle #(•) is constantly equal to a(Ao) := 9q — ctAq on the 
interval [2Xq,T — 2Ai], we deduce that the two curves coincide, up to an appropriate choice of T > 0, if and 
only if the angle /3(Ao) between the vector P\ — Pq and the horizontal axis is equal to a(Ao) (up to a multiple 
of 2tt). 

Let us observe that A + Ai < y^Sn/a + yjbir/a < Ay/ir/a. Since |Po| < 2A and \P\ — (x\,y\)\ < 2Ai 
when Ao 7^ and Ai 7^ 0, if |(aii,yi)| > 8^/tt/(j then 

((xx,yi),Pi - Po) = ((x 1 ,y 1 ),P 1 - (xi,yi)) + ((xi,yi), (xi,yi)) - ({x 1 ,y 1 ),P a ) 
> |(a; 1 ,y 1 )| 2 -2(Ao + A 1 )|( a;i , 2 / 1 )| 
0. 
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Figure 7: Proposition 14. 11 construction of the reference trajectory 



This implies that /?(•) takes values on an interval of length less than it. Moreover, since Pq and P\ are 
continuous functions of Ao, the map /?(•) is also continuous. It follows that the range of the continuous 
function a(-) — /?(•) is an interval whose length is larger or equal than 2tt and therefore it contains a multiple 
of 2-7T, i.e. there exists Ao <G [0, yjiir/a] such that a(Ao) = /3(Ao) up to a multiple of 27r, as required. □ 

Comparison with the reference trajectories defined above leads to relevant estimates as shown by the following 
proposition. 

Proposition 4.2. There exists a constant only depending on ip,ip such that the following holds: if 

|(xi,yi)| > 8-^/7? and if u opt {-) is an optimal control defined on [0,T] steering the system from Xq to X\ = 
(xi, yi, 9\, 0) the following relations hold 

\{xi,Vi)\ < T < C(u opt (-)) < |(si,yi)|+C^. (4.2) 

Consequently, 

(<p(n(t)) + ip{Uop t {t))) dt < . (4.3) 

Proof. If |(:£i,yi)| > Sy/n/a we know from Proposition 14.11 that there exists u(-) defined as in (|4.1I) and 
steering the system from Xq to X\ . If Pq , Pi are defined as in the proof of Proposition 14.11 we have the 
following estimates 

C(u(-)) = T + 2(Ao + Ai)^( ( t)+2(/ Ao <p(at)dt + f* 1 ip(at)dt) 



< 



t + 8^(^(0-) + y\\ LOO([0tVE ^ } ) 

= |Pl - Pol + 8v^(l + + |MLoo ([ o,V5^)) 

< 1(11,1/01 + \Pi - (»i,I/0l + l^ol + Sy/£ft(l + + IMIi.Ho.vsss])) 
\(xi,yi)\ +8^/7/^(2 + xP((t) + ||vllioc ([ o,V5¥?])) ■ 



< 

Thus, by choosing a — 1 and since C(u opt (-)) < C(u(-)), we deduce the explicit bounds (|4.2|) . (|4.3[) . with 
= 8v^(2 + + ||^|lioo ([0i V5S])) ■ D 

Remark 4.3. For every e > and every optimal control u op t defined on [0,T], let U eps be the subset of [0,T] 
given by 

U e = {te [0,T] : \u opt (t)\ >e}. 

From Equation (|4.3[) and the strict convexity of we deduce that for every e > there exists a positive 
constant C v ^, such that for every u opf defined on [0,T], fi{lA e ) < C v ^. 

As a consequence of (|4.3p we easily get the uniform equicontinuity of the k components of the optimal 
trajectories, solutions of (OCP). This is a particular case of the following lemma, that will also be useful in 
the next sections. 



13 



Lemma 4.4. For every T > and e > there exists <5 £! r > such that \si — s 2 \ < <5e,r implies \k(si) — 
K ( s 2)\ < £ /or every [si,s 2 ] C [^1,^2]; whenever t\,t 2 and u(-) are such that vp(u(s))ds < T. Moreover 



-00. 



Proof. By using condition (H3) and 

\k(s 2 ) 



< 



< 



< 



< 



< 



si 

•S2 



we get 
u(r)\dT 
f" ((\u(r)\-R) + +R)dr 
((\u(r)\-R)+) P dT 

S2 

4>{u(T))dT 

I 

Iv 

1/9 



1/p 



32-8^1" 



1/p 



s 2 - Si 



1/9 



- R\s 2 - si| 
R\s 2 - si| 



\s 2 - Sl| 



R\s 2 - si 



(4.4) 



where, for a real valued function /(•), we define f + (t) = max{0,/(t)} and q is such that l/p+ l/q = 1. 
To conclude the proof of the lemma it is enough to observe that (H3) actually holds for arbitrary small R, 
provided that C is also chosen small enough. □ 

Comparisons with reference trajectories also give estimates of the cost of pieces of trajectories that are 
close to a line segment. 

Lemma 4.5. For every C > there exists 5 > and R > large enough such that the following holds. Let 
Wo = (xo, ?/0j 9o, kq), W\ = (£1,2/1, 0i, «i), and set {x\ — xo,y~i — yo) = r(cos0,sin(?) for some T > and 
9 £ [0, 271"]. Then, if \9i — 9\ < 5, |«,-| < S for i = 0, 1 and T > R, any optimal trajectory connecting Wq to W\ 
satisfies C{u(-)) < \(xi - x ,y~i - yo)\ + C. 

Proof. The proof of the lemma relies on the construction of a special trajectory satisfying the hypotheses of 
the lemma, for suitable values of 5 and R, and such that C(u(-)) < \(x\ — xo,yi — y~o)\ + C. Consider control 
functions of the form: 

5 t e [0, n] , 
-5 t e (n,T 2 ] , 

u(t)={ te(r 2 ,T-T 3 ], 

51 t e (T - r 3 ,T - t 4 ] , 
-5i t £ (T — 74, T] , 

for some choices of Tj > , i = 1, 2, 3, 4, of T > and with Si € {<5, — <5} for i = 0, 1. Let us consider the 
two trajectories 7°(-) = (x° (■) , y° {■) , 9° (■) , k° (■)) and 7 1 (-) = (x 1 (■) , y 1 {■) , 9 1 (■) , n 1 (■)) corresponding to u(-) 
and such that 7°(0) = Wo and 7 1 (T) = W\, respectively. If we suppose that t 2 = 2ri + no/So then a simple 
computation shows that K°(t) = on [T2, T — T3]. We compute the value of $(•) on [t2,T — T3] 



= 9o + So 



, no 
00 



2S 



t e [t 2 ,T-t 3 ] , 



and we observe that, for every fixed 9o, kq satisfying \9q— 8\ < S and \ko\ < S and for every value 9 £ [0 — 6, 9+6] 
there exists 6 £ {-6, 6} and n £ [0,1 + y/5/2] such that r 2 = 2n + k /5 € [0, 1 + \/T0], = <?. Therefore 
we can associate to each value 9 £ [9 — 6,9 + 6] a value of t 2 and, as a consequence, we can construct a 
continuous map associating 9 £ [9 — 6, 9 + 6] to a point (x°(t 2 ), y (r 2 )). 

Let us now consider the trajectory 7 1 (-). If we set T3 = 2t 4 + K1/61 we have that K 1 (t) = on [72, T — 73]. 
Moreover we have 



9(t) = 9 X - Si 





Ki\ 2 


«?1 


(r 4 H 




2oi. 



*e [r 2) T-r 3 ] 



Again, it is possible to associate to each value 6 £ [9 — 6,9 + 6] corresponding values of 61 £ {—6,6} and 
< 14 < 73 < 1 + \/T0 in such a way that (x 1 (T — 73), y 1 (T — r 3 )) varies continuously with respect to 9. 
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We want now to prove that 7°(-) and 7 1 (-) coincide, up to the choice of T, for a suitable value of 8. 
Since it is easy to see that (x°(t2), V°{t2)) = (xq, yo) + r 2 (cos#, sin#) + Oi(5) and (x l (T — r 3 ), y 1 (T — r 3 )) = 
(xi, yi) — t 3 (cos(9, sin 6) + 02(6) with |Oj(5)| < MS for some universal constant M we have that 

(x\T - n),y\T - r 3 )) - (x (T2),y (T 2 )) = (T-T2- t 3 )(cos8, sm6) - 1 (5) + 2 (S) 

= r'(cos0',sin0') 

for some 8' with \0' — 8] < 5 and r" > 0, provided that R is large enough (independently of 5). In particular 0' 
can be thought as a continuous function of 9, and we conclude that 9' — 9 = for some value of 9 € [8 — 6,8 + 5]. 
This implies that 7°(-) = 7 1 (-)i U P to choosing T = V + t 2 + r 3 . 

We have therefore constructed a trajectory (x(-), y(-), &(■), «(•)) = 7°(-) = 7 1 (-) corresponding to u(-) and 
connecting Wo to Wi- 

Let us estimate the cost corresponding to this trajectory. We have that 

£ tew)) + ^{u{t)))dt < 2(1 + v^HiMi^ ^^^ + m) , 

where the right-hand side can be made arbitrarily small by appropriately choosing 5. It remains to compare T 
with the difference |(xi — xq, y~\ — yo)]. Wc know that ]{x\ — Xo, yi — yo)] = Jq cos(0(i) — 8)dt. Since \6(t) — 8\ < 
AI5 for a suitable M > wc have that J^ 2 cos(8(t) - 8)dt > r 2 (l - M 2 5 2 /2) and that /^ ^ cos(0(i) - 8)dt > 

t 3 (1 — M 2 6 2 /2). Moreover T3 cos(8(t) ~ 8)dt is the projection of the segment between P\ = (x(T2),y(r2)) 
and P 2 = (x(T — r 3 ),y(T — t 3 )) on the line / connecting (xo,yo) to (xi,y~i). In particular since the distance 
among the points Pi and the line I is bounded by M6 for a suitable M > 0, it is easy to verify that if 
T — t 3 — T2 > 4M 2 then the difference among the length of the segment between Pi and P2 and the length of 
its projection on I is bounded by 6 2 . In other words Ti cos(8(t) — 8)dt > T — r 3 — t 2 — 5 2 . By choosing i5 
small enough we then conclude the proof of the lemma. 

□ 

4.2 Some preliminary lemma 

Let a £ [0,27r) be such that (xi,y±) = \(x\, yi)|(cos a, sin a) and let us write as (pi,P2) = p(cos<^, sin</>), for 
some 4> S [0; 2?r), the first two components of the covector associated to an optimal trajectory and by 8{-) the 
corresponding angle. Note that the evolution of p 3 is described by the equation 

p 3 (f)=psiii(0(t)-0). (4.5) 

We have the following lemma. 

Lemma 4.6. For every e > there exists T e > such that, for every optimal trajectory, one has p-(J £ ) < 71, 
where the set J £ is defined as 

J e = {re]0,T] : \a-8(r)\>e}. 



Proof. Let us first note that = ((cos a, sin a), (xi,yi)}. Since (x±,yi) = J Q (cos 8 (t), sin 8 (t))dt we 

therefore get 

](x 1 ,y 1 )\= / cos(a - 8{t))dt . 
Jo 

In particular if e is small enough we have 

\(xi,yi)\ = J cos(a - 8(t))dt + J cos(a - 8(t))dt < (l - j)/i(J e ) + v{J c e ) = T — £ —^{J £ ). 

Since from the uniform bound (|4.2p we have T — < \(xi,yi)\ for |(xi,yi)| large enough the lemma is 

proved with T E = 4C ¥ , i ^,/e 2 . □ 

Lemma 4.7. Fix n > 1 and k > and, for any optimal trajectory and any k > k define Sk = {t £ [0,T] : 
\n(t)\ £ [k,krj\\. Then there exists C v ^ > 0, independent of k, such that p(Sk) < C v ^. 
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Proof. Note that Sk is the union of two sets S~£ and defined respectively by the sign of k. 

For the sequel, we only provide estimates for S~£ since the same ones hold true for . For 5 > 0, let Ks,k 
be the union of all disjoint subintervals of S£ of length at least 5. We first prove that there exists ^ > 
such that /i(Ks : k) < ^ for any optimal trajectory and k > k. 

Assume without loss of generality that 6 < 2ir/k and fix e = 5k/ 3. Notice that for every measurable set 
5c5j!" such that J s 9(t)dt = J s n(t)dt = I for some I > one has fi(S) > l/{krj). 

It turns out that if [ii,i2] C S~£ is such that n(t)dt = 3e then /ifjii,^] H J s ) > e/(krf), where J s is 
defined as in Lemma [4.61 Clearly t% — t\ < 3e/k < 3e/k = 5. Let now I C be an interval such that 
M(-0 > <5 an d m > 1 be the largest integer such that 3em/k < /x(J). Since jti(J) < 3e(m + l)/k < 6em/k it 
turns out that 

Thus, by definition of Kg,k and the previous inequality, one gets 

T s > n(J g ) > ii{K 5>k n j e ) > p 



hence proving that the measure of Kg t k is bounded by Cf. ^ = 6ifT E 



6rj 

v,4> 



To conclude the proof of the lemma let us define — {t e [0, T] : n(t) £ [k/u, kr]f]} for some v > 1 and 
define Ks,k similarly as before. Let us observe that [k/v,kr)v\ D [k, krj\ + -8(0, £*) for some > and for 
every k >k. According to Lemma 14.41 there exists 6+ such that | k (s) — k (t) | < e* if | s — 1 1 < 5* for s , t G [0, T] . 
One deduces that C Ks^.k- Since, as proved above, Ks*,k has uniformly bounded measure, we get the 
conclusion. 

□ 

Lemma 4.8. There exists a constant > such that for any optimal trajectory ||k(£)||l°°([o,t]) ^ ^<p,f- 
Moreover, for every e > 0, there exists T e > such that, for every optimal trajectory, one has /J,(K e ) < T e , 
where the set K e is defined as 

K e = {tE [0,T] : \u(t)\ > e or |«(r)| > e} . 



Proof. Fix r) > 1. Given an optimal trajectory, define k = ||K(i)|| i oc([ ,T])/ ? 7- Since k is continuous and 
k(0) = k(T) = 0, the inverse image of [k, krj] under n contains a segment [s, t] such that \n(s) — n(t)\ = k(r]—l). 
From Lemma 14.71 there exists a positive constant independent of k such that \s — 1\ < C Vj ^. Therefore, 
the uniform equicontinuity characterized in Lemma 14.41 for, more directly, Equation (|4.4|l ) yields a uniform 
bound on \k(s) — n(t)\ and thus on k. The first part of the lemma is proved. 

As regard the second part of the lemma, by using Remark 14.31 it is enough to prove that the set C £ := 
{t G [0, T] : \k(t)\ > e} has uniformly bounded measure. This easily follows from Lemma [4.71 by noticing 
that C £ = {t g [0,T] : \k(t)\ G [e,r]e]} with r] = T v4 ,/e. 

□ 

Remark 4.9. In the case where the curvature at the extremities is free, the bound on ||fs(i)||i°°([o,T]) on ly holds 
when T is greater than a positive constant T v4 , > 0. Indeed, at the beginning of the proof of the lemma, the 
existence of a segment [s,t] such that \k(s) — K(t)\ = k{i] — 1) can not be ensured for any trajectory when k(0) 
and k(T) are not equal to 0. However such a segment exists as soon as T is greater than the constant C Vi ^ 
given by Lemma 14.71 The rest of the proof is unchanged. 

We need now a simple technical lemma. 

Lemma 4.10. For every m G N there exist (j, m > and X m G (0, 1) such that, for every t\,ti G R, C > 
and f G C m {[ti,t2\) satisfying |/' m ^(i)| > C there exists a subinterval of \b\,t2\ of length X m {t2 — t\) where 
\f(t)\ > fJ, m C(t2 — ti) m . In particular we can take A m = 4 _m and fi m = 4 . 

Proof. The proof goes by induction. Clearly the lemma is true for m — 0. Assume that the lemma is true 
up to m = m and let us prove it for m = rh + 1. By the inductive hypothesis we know that |/^ m+1 '(£)| > C 
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implies that |/'(t)| > /x m C(t2 — t\) m on a certain subinterval of [^1,^2] of length Am (£2 — ti)- For simplicity, 
and without loss of generality, let us identify this subinterval with [0,r], where r = A m (t2 — t\), and let us 
assume that / is increasing on it. 

Wc claim that |/(t)| > ^ x ^ c (t 2 - tx)" l+1 either in [0, r/4] or in [3r/4, r]. Indeed if this is not true then, 
since / is increasing, we would have /(r/4) > - &™±&£ (t 2 -ti)™ +1 and /(3r/4) < ^ (t 2 which 
implies /(3r/4) - /(r/4) < ^'^ c ft 2 - = /i m C(t 2 - *i) T "§ and which contradicts the hypothesis 

|/'(t)| > /i m C(t2 — t\) m ■ The lemma is therefore proved by taking A m +i = A m /4 and /U m +i = /i m A m /4. 

□ 

As a consequence, we obtain a bound on the L°° norm of (p3,Pi). 

Lemma 4.11. There exist two positive constants and T v ^ such that, for every optimal trajectory defined 
on [0,T] with T > T v ^, one has ||(j>3,J>4)||l~([o.t]) < C<p,i>0- + p), where we recall that p := |(pi,P2)|- 

Proof. Define ji := pi/(l + p) for / = 3, 4. Then one has 

73 = sin(0 - (j>), 74 = -73 + + p), 

along any optimal trajectory defined on [0, T]. Notice that |73(t)| < 1 and, from Lemma 14.81 we have 
|<//(k)|/(1 + p) < Ktp^, for some constant K v _^,. From the dynamics of 74, if to is such that (^o) I = 
ll73|U°°([o,T]), on c gets that |-y 4 (*)| > 1 1 T3 1 1 ([o,t]) ~ I* ~ *o] - K v ^ for t e [0,T]. From Lemma S7TU1 we 
deduce that, if ||73||i°°([o,T]) is n °t uniformly bounded for T large enough then equation (|4.3[) cannot hold 
true. Therefore we get a uniform bound on I74I which, still according to (|4.3p and a standard argument by 
contradiction, implies the uniform boundedness of 74. The proof of the lemma is complete. 

□ 

We use the previous lemma to obtain the following one. 

Lemma 4.12. For every v > there exists T v _^ > such that, for any optimal trajectory defined on [0,T] 
with T > T v ^, one has p > 1 — v. 

Proof. We argue by contradiction and assume that | Cpi , P2 ) | < \ — v. For e > and to <G [0, T] \ K £ , where 
K £ is defined in Lemma 14.81 the Hamiltonian at time to writes 

H = p 1 cos6(t ) +P2sm9(t ) + P3(*o)k(*o) +P4(t )u(t ) - (1 + y(«(*o)) + ^("(*o))) 

< -l + |(pi,p 2 )|+2C^e 

< —v + 2C v tfE. 

Take now e = v jAC v ^,, and T > p(K £ ) in the contradiction assumption, so that [0, T] \ K £ ^ 0. One therefore 
gets that H < 0, which contradicts the PMP. Hence the conclusion. 

□ 

The next proposition will be crucial in order to prove our main asymptotic results. 

Proposition 4.13. For every rj > there exists R v > such that for every optimal trajectory with \(xi,yi)\ > 
Rrf one has \<f> — a\ < rj and p < 1 + rj. 

Proof. We first establish the fact that \<j> — a\ < 77 if |(xi,yi)| is large enough. Assume that \tj> — a\ > rj. 
Then, our aim is to prove that there exists T v such that T < T n . One has, for every t, s € [0, T], 

\e(t)-e(s)\ <c v ,4\t-s\, 

where C vi p is provided by Lemma |4~I51 Set e = r)/3 and S £ — e/C^.^p. Let J e be the set defined in Lemma I4TB1 
and consider the set 

J £ := (J 2e + B(0,6 £ )) n[0,T]. 

Clearly, from the definition of 5 £ , we have that J £ C J £ . Therefore from Lemma T4. 61 we have that p,(J £ ) < T £ . 
Moreover for every t <G J e \ ([0, 5 £ ] U [T — S E ,T]) there exists a neighborhood in [0, T] of diameter at least 2S £ 
and completely contained inside J £ . We deduce that J £ is the union of a finite number of intervals and that 
the number of these intervals is bounded by T £ /{28 £ ) + 1. As a consequence the complement J £ is also the 
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union of at most n £ := T £ /{25 £ ) + 2 <T £ /S £ intervals. Since jtt(Jg) >T — 7^>-jifTis large enough, we get 
the existence of a subintcrval I of J £ such that 

M/)>^>P=:M S T. 
n e 2T £ 

In particular, since J £ C J| e implies that \<f>— 9(t)\ > \<p— a\ — \a — 0(t)\ > e on I, we have that, if \(x±, yi)\ 
is far enough from the origin, one gets according to (|4.5I) and Lemma 14. 121 

\p 3 \ > ^sine, on /. (4.6) 

By Lemma [4~T0l there exists a subinterval I' of I with jtx(I') > M £ T/4 and |p 3 | > N £ T on where N £ 
only depends on e. We immediately deduce from Lemma 14.81 and the equation of p^ that \p^\ > N e T/2 on 
I' for T large enough. Again, by Lemma [4.101 there exists a subinterval I" of I' with p(I") > M e T/16 and 
\p4\ > N £ M £ T 2 /16 on 7". That contradicts Remark 14.31 and we deduce that |0 — a| < r\ for |(a?i,yi)| large 
enough. 

As for the second one, this is an immediate consequence of the above results. Indeed, for every e > 0, there 
exists T large enough such that every optimal trajectory defined on [0, T] with T >T satisfies \0(to) — <p\ < e, 
|/c(to)| < £ an d |"(*o)| < £ > f° r some t n G [0,T]. Then, from H = 0, we get 

= pcos(e(t )-(l))+p 3 (t )K(t )+p A (t )u(t )-(l + (p(K(to)) + ^(u(t ))) 
> p(l - e 2 /2) - 2C V ^(1 +p)e-l- ||v>QIU»([o >£ ]) - V(e) 
= P (l - 2C v ^e - e 2 /2) - 1 - 2C v ^e - ||^(-)IU~([o, e] ) - 1&(e), 
and, by taking e small enough, the proof of the lemma is concluded. 

□ 



4.3 About Propositions [3H and [3761 

One immediately deduces from Lemma 14.111 and Proposition 14.131 that p 3 , pi and u are uniformly bounded 
for T large enough over all optimal trajectories. This, together with Lemma 14.81 establish Propositions 13.41 
Proposition 13.61 results from Lemma [4.121 and Proposition ^. 131 

At first sight it seems reasonable to conjecture that the previous results can be improved in the following 
directions: 

(a) extending the uniformity results to all optimal trajectories, i.e. independently of the final time T; 

(b) as the terminal point {xx,y{) goes to infinity, the corresponding optimal control u(-) tends to 0. 
However, the next two remarks show that it is not the case. 

Remark 4.14. Disproving Conjecture (a) amounts to show that the control function u(-) associated to optimal 
trajectories reaching points in a neighborhood of the origin is not uniformly bounded. More precisely, we will 
exhibit a sequence of points x[ n ^ = (xj™ , y[ n \ # x , 0) with lim^oo K^j ,2/x )l = such that the optimal 
controls u^(-) steering the system from Xq to x[ n ' satisfy lim„_ ! . 00 ||u^(')||oo = °°- F° r £ > 0, let us consider 




te[0,e/2], 
te(e/2,e], 



where u e > verifies etp(u £ ) = m, where m is a positive constant to be fixed later. Note that u £ tends to 
infinity as e goes to and u e < C<y(m/ e) x t p for Co independent of e and m, provided that e/m is not too 
large. It implies that (p e := (p(u E s)ds tends to as e goes to 0. 

We remark that, for every t € [0,e] the angle 9 £ (t) corresponding to u e (t) maximizes 0(t) under the 
constraints |ju||oo < u e , 9(0) — 7r/2, k(0) = and n(e) = 0. Indeed, for every such u(-), one has k(£) = 
J Q u(s)ds < J Q u e (s)ds, for every t £ [0,s]. Notice also that 9 £ (e) = ir/2 + u £ e 2 /4. Assuming that we start 
from the origin we also have that 

y(t)= [ sm6(t)dt> [ sm6 £ (t)dt = y £ (t) and x(t) = [ cos 9 (t) dt > f cos6 £ (t)dt = x £ (t) 
Jo Jo Jo Jo 
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where 9(-) is associated with a control u(-) satisfying the previous constraints and not almost everywhere equal 
to u e (-). This implies that, in order to reach the point (x e (e),y e (e)) at time T with a control «(•) ^ u e (-) 
satisfying the previous constraints, it must be T > e and y(e) > j/ e (e) so that there exists t £ [s,T] such that 
y(t) < 0, i.e. 9(t) € (ir, 2ir). Let us prove that the corresponding trajectory cannot be optimal. First we note 
that the total cost corresponding to u £ (-) is m + e + 2ip £ . Therefore, if we assume by contradiction that u(-) 
is optimal we must have T < m + s + 2<y9 e and J * ijj(u(T))dT < m + e + 2<y9 e for every £ £ [0, T]. From Holder 
inequality, we deduce that \ K {t)\ < C 1 [m + e + 2f e } 1 ^t 1 - 1 /P, which implies t hat \0(t) - n/2\ < d [m + e + 2^ e ] 2 , 
for some C\ > independent of m and £ small enough. By taking m = ^Jtt/ (3Ci) we reach a contradiction. 
Therefore, any optimal control connecting X to X\ = (x e (e),y E (e),9 E (e),0) must satisfy ||ii||oo > u e for e 
small enough. 

Remark 4.15. We next show that Conjecture (b) is false by disproving that, for every e > 0, there exists R e 
such that |ju||oo < £ for every optimal triple (X(-),u(-),T) with |(ari,j/i)| > R £ . Indeed, if a is defined as in 
Section H^and \(xi_, yi)\ is large enough the previous results say that pi is arbitrarily close to sin a, and thus 
different from 1 in general. On the other hand, H = and 6(0) = ir/2 imply u(0)ip' (u(0)) — ip(u(Q)) = 1 — P2 
and therefore u(0) is not close to in general. 

4.4 Proof of Theorem 13.71 

We will prove the theorem by showing separately that the functions &(■) — ct, n(-),p3(-),p^(-) can be made 
arbitrarily small by choosing large enough r u ,a u . 

From Lemmas 14.61 and 14.81 we have that, given 8' > and if T > 27a' for a suitably large Ts 1 , there exists 
to e [0,Ts'] and t x S [T - Ts> ,T] such that, \a - 6(U)\ < 8' and \6(U)\ < 8'. If we set W = X(t ) and 
Wi = X(ti) and we let 9 be as in Lemma l4~5l then, if T is large enough (depending only on 8'), we have that 
I a — 9\ < 8' and the hypotheses of Lemma H. 51 are satisfied with 8 = 28'. From Lemma T4.51 and for a fixed 
C > 0, we have that the optimal control u(-) must be such that J t 1 ijj(u(t))dt < C, if 8 is small enough. 

Let us fix e > 0. If C is small enough then, from Lemma 14.41 we can assume that \s\ — sa| < 47^ 
implies |/c(si) — K ( s 2)\ < e := 7^, where 7^ is defined by Lemma 14.61 By contradiction it is then easy to see 
that || «(•) || i<» ([ti,ta]) < 2e if T is large enough. Indeed, otherwise we would have |«(t)| > £ on an interval 
of length larger than 47^ and then, as a consequence of Lemma [4.101 we would get \9(t) — 9\ > e on an 
interval of length larger than % contradicting Lemma [4.61 Moreover, since ||K( - )lli oo ([ti,t 2 ]) — ^£ we have 
l|0(") — ^llL=°([ti,t 2 ]) — £ + 2£/7^, again as a consequence of Lemma 

It remains to prove that ||(p3(-)jP4('))lli oc ([ti,t9]) can De made arbitrarily small by an appropriate choice of 
r Vl a v . For this purpose it is enough to observe that ||.P3||i,°o([i 1 ,i a ]) can be made arbitrarily small, and, again, 
a simple argument by contradiction based on Lemma 14.101 and Remark 14.31 leads to the conclusion. 

□ 
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